Stochastic Games and Related Classes

For n-person perfect information stochastic games and for n-person stochastic games with Additive Rewards and Additive Transitions (ARAT) we show the existence of pure limiting average equilibria. Using a similar approach we also derive the existence of limiting average "-equilibria for two-person switching control stochastic games. The orderreld property holds for each of the classes mentioned, and algorithms to compute equilibria are pointed out. 1 Model We deal with n-person stochastic games with nite state and action spaces. Only for some special classes of stochastic games limiting average "-equilibria are known to exist, but generally their existence remains to be an open problem (see Thuijsman 1992] for a survey on equilibrium existence). For zerosum stochastic games Mertens & Neyman 1981] showed the existence of the limiting average value. Approaching this value generally involves the use of history dependent strategies. In this paper we show existence of limiting average equilibria (inàlmost stationary' behavior strategies) for perfect information stochastic games and for stochastic games with Additive Rewards and Additive Transitions (ARAT). For two-person stochastic games with switching control we show the existence of limiting average "-equilibria. For none of these related classes (any perfect information game has ARAT as well as switching control structure) equilibrium existence was known before. Our method implies that the orderreld property holds for these classes, i.e. if payoos and transitions are rational, then there are rational equilibrium strategies and rational equilibrium rewards as well. Algorithms to determine equilibria are also pointed out. Whenever we speak about equilibria, optimal strategies, best replies etc., we shall always have limiting average rewards in mind. Pure stationary optimal strategies exist for zerosum perfect information games (cf. Liggett & Lippman 1969]) as well as for zerosum ARAT games (cf. Raghavan et al. 1985]). For zerosum switching control stochastic games there are also stationary optimal strategies (cf. Filar 1981]), these however are not necessarily pure. An example at the end shows that in the non-zerosum case stationary solutions may fail to exist in any of the classes mentioned. Another example explains why for switching control games we have to restrict to the two-person case.